
^9/980130 

JCOSRec^d PCT/FTO 2 8 NOV 2001 



DESCRIPTION 
INFORMATION PROCESSING APPARATUS 
Technical Field 

The present invention relates to an 
information processing apparatus, in particular, to an 
information processing apparatus that can execute a 
plurality of signal processes. 
Background Art 

A signal processing device disposed in a 
signal' processing apparatus is normally structured so 
that one function is accomplished. 

In other words, to accomplish a plurality of 
different signal processes, it is necessary to provide 
devices corresponding to the number of required 
processes. Thus, for example, it was difficult to 
compactly structure and reduce the cost of the 
apparatus . 

Disclosure of the Invention 



above-point of view. According to the present 
invention, corresponding to a predetermined function 
execution request, the internal structure and process 
contents of the same hardware are changed so as to 
execute a plurality of processes. 



present invention, comprising a plurality of 
classifying adaptive processing circuits for performing 



The present invention was made from the 



An information processing apparatus of the 



a classifying adaptive process for input information 
signals, and a switching circuit for switching a 
connection relation among the plurality of classifying 
adaptive processing circuits. 
5 According to the information processing 

apparatus of the present invention, a plurality of 
classifying adaptive processing circuits perform a 
classifying adaptive process for a desired function 
corresponding to an input information signal. The 
10 relation of the connections of the plurality of 

3 classifying adaptive processing circuits is switched. 

3 At least one of the classifying adaptive 

3 processing circuits is configured for switching the 

g corresponding classifying adaptive process for the 

"l5 corresponding information signal as the connection 

T relation of the switching circuit is switched. 

% At least one of the plurality of classifying 

^ adaptive processing circuits is configured for 

switching the structure of the corresponding 
20 classifying adaptive process as the connection relation 

of the switching circuit is switched. 

The structure represents a structure of class 
taps or a structure of predictive taps. 

At least one of the plurality of classifying 
25 adaptive processing circuits is configured for 

switching a coefficient of the corresponding 
classifying adaptive process so as to switch the 



process for the corresponding information signal as the 
connection relation is switched by the switching 
circuit . 

The input information signals are output 
through the plurality of classifying adaptive 
processing circuits . 

A pre-processing circuit is configured for 
switching a predetermined process as the connection 
relation is switched. The pre-processing circuit is 
configured for performing a predetermined process for 
the corresponding input information signal and 
inputting the processed result to the classifying 
adaptive processing circuit. 

A post -processing circuit switches a process 
as the relation of the connections is switched. The 
post -processing circuit performs a predetermined 
process for an output of a classifying adaptive 
processing circuit. 

Examples of processes that the classifying 
adaptive processing circuits accomplish are as follows. 

The information signals are picture data 
composed of pixel information. One of the plurality of 
classifying adaptive processing circuits is configured 
for performing the classifying adaptive process 
corresponding to the pixel information of the 
corresponding input information signal and predicting 
pixel information that has to be present between the 



pixel information of the input information signal and 
pixel information adjacent thereto so as to improve the 
resolution of the picture data. 

The information signals are picture data 
5 composed of pixel information. One of the plurality of 

classifying adaptive process circuits is configured for 
performing the classifying adaptive process for the 
corresponding input information signal using a prepared 
left eye coefficient and predicting pixel information 
10 of left-eye picture data and for performing the 

£3 classifying adaptive process for the corresponding 

input information signal using a prepared right- eye 

O coefficient and predicting pixel information of right - 

eye picture data so as to generate stereo picture data 

^;"'15 with the left -eye picture data and the right -eye 

l2 picture data. 

[U The information signals are picture data 

C3 composed of pixel information. One of the plurality of 

classifying adaptive processing circuits is configured 
20 for performing the classifying adaptive process for the 

corresponding input information signal using a prepared 
luminance signal coefficient and predicting a luminance 
signal component of the picture data and another one of 
the plurality of classifying adaptive processing 
25 circuits is configured for performing the classifying 

adaptive process using prepared color difference signal 
coefficients and predicting color difference components 
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of the picture data so as to separate the picture data 
into the luminance component and the color difference 
components . 

The information signals are picture data 
5 composed of pixel information. At least two of the 

plurality of classifying adaptive processing circuits 
are configured for performing the classifying adaptive 
process for the pixel information having different 
phases and changing the number of pixel information 
10 that composes the picture data. 

.... The information signals are picture data 

composed of pixel information. At least two of the 
jzf plurality of classifying adaptive processing circuits 

are configured for performing the classifying adaptive 
la. 5 process and obtaining a plurality of picture data 

H having different resolutions corresponding to the 

classifying adaptive process performed by the plurality 
n of classifying adaptive processing circuits. 

One of the plurality of classifying adaptive 
20 processing circuits is configured for performing the 

classifying adaptive process for the corresponding 
input information signal and obtaining picture data 
having a first resolution and another one of the 
plurality of classifying adaptive processing circuits 
25 is configured for performing the classifying adaptive 

process for picture data having the first resolution 
and obtaining picture data having a second resolution. 
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The information signals are picture data 
composed of pixel information and structured in the 
unit of a frame. One of the plurality of classifying 
adaptive processing circuits is configured for 
5 performing the classifying adaptive process for the 

corresponding information signal that is input in the 
unit of a frame and generating picture data of frames 
chronologically preceded and followed by a frame of the 
input information signal. ~ 
10 Brief Description of Drawings 

CP Fig. 1 is a block diagram showing an example 

of a basic structure of a data processing apparatus 
O according to the present invention; Fig. 2 is a block 

Uj diagram showing an example of the structure of a pre- 

ri5 processing portion of the data processing apparatus; 

Figs. 3A and 3B are block diagrams showing an example 
■L^ of the structure of a data processing portion of the 

H data processing apparatus; Fig. 4 is a block diagram 

showing an example of the structure of a post- 
20 processing portion of the data processing apparatus; 

Fig. 5 is a block diagram showing an example of the 
structure of a data processing apparatus 1 that 
executes a process for increasing the resolution in 
process mode (1); Fig. 6 is a schematic diagram showing 
25 pixel arrangements of an SD picture and an HD picture; 

Figs. 7A and 73 are block diagrams showing an example 
of the connections of the data processing portion in 
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the process mode (1); Fig. 8 is a block diagram showing 
an example of the structure of a classifying adaptive 
processing portion that performs a prediction process; 
Fig, 9 is a schematic diagram showing an example of 
5 class tap extraction information in the process mode 

(1); Fig. 10 is a schematic diagram showing other pixel 
arrangements of an SD picture and an HD picture; Fig, 
11 is a block diagram showing an example of the 
structure of a learning device J:hat performs a learning 
10 process for calculating predictive coefficients that 

are pre- stored in a coefficient memory of a classifying 
% adaptive processing portion; Fig. 12 is a block diagram 

showing an example of the structure of a data 
processing apparatus that performs another processing 
^3l5 method for the process mode (1); Figs. 13A and 13B are 

block diagrams showing an example of the connections of 
the data processing portion that performs the other 
processing method for the process mode (1); Fig. 14 is 
a schematic diagram showing an example of class tap 
20 extraction information in the other processing method 

for the process mode (1); Fig. 15 is a schematic 
diagram showing a pixel arrangement in the other 
processing method for the process mode (1); Fig. 16 is 
a block diagram showing an example of the structure of 
25 the data processing apparatus that executes a process 

for generating a picture dedicated for a left eye and a 
picture dedicated for a right eye in process mode (2); 



Figs. 17A and 17B are block diagrams showing an example 
of the connections of the data processing portion in 
the process mode (2); Fig. 18 is a schematic diagram 
showing an example of class tap extraction information 
in the process mode (2); Fig. 19 is a schematic diagram 
showing in more reality predictive taps in the process 
mode (2); Fig. 20 is a schematic diagram for explaining 
a picture that is learnt; Fig. 21 is a block diagram 
showing an example of" the structure of the classifying 
adaptive processing portion that executes a learning 
process in the process mode (2); Fig. 22 is a block 
diagram showing an example of the structure of the data 
processing apparatus that executes a process for 
generating a luminance signal and color difference 
signals in process mode (3); Figs. 23A and 23B are 
block diagrams showing an example of the connections of 
the data processing portion in the process mode (3); 
Fig. 24 is a block diagram showing an example of the 
structure of the data processing apparatus that 
accomplishes a Y/C separating device that separates a 
composite signal NTSC into a component signal (Y, U, V) 
corresponding to the classifying adaptive process in 
the process mode (3); Fig. 25 is a schematic diagreim 
showing an example of class tap extraction information 
in the process mode (3); Fig. 25 is a schematic diagram 
showing an example of the predictive tap extraction 
information in the process mode (3); Fig. 27 is a block 
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diagram showing an example of the structure of the 
classifying adaptive processing portion that executes a 
learning process in the process (3) is executed; Fig. 
28 is a block diagram showing an example of the 
5 structure of the data processing apparatus that 

converts the aspect ratio in process mode (4); Figs. 
29A and 29B are block diagrams showing an example of 
the connections of the data processing portion in the 
process mode (4); Fig. 30 is a schematic diagram for 
10 explaining the classifying adaptive processing portion 

f^. of the input data processing portion that performs a 

predictive process in the process mode (4); Fig. 31 is 
^ a block diagram showing an example of the structure of 

the data processing apparatus that generates a picture 
Ol5 having a different resolution in process mode (5); Figs, 

H 32A and 32B are block diagrams showing an example of 

fy the connections of the data processing portion 12 in 

n the process mode (5); Fig. 33 is a block diagram 

showing an example of the structure of the data 
20 processing apparatus that converts a frame rate in 

process mode (6); Figs. 34A and 34B are block diagrams 
showing an example of the connections of the data 
processing portion in the process mode (6); Fig. 35 is 
a schematic diagram for explaining frame structures of 
25 an input picture and an output picture; and Fig. 36 is 

a schematic diagram showing an example of class tap 
extraction information in the process mode (6). 
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Best Modes for Carrying out the Invention 

Next, with reference to the accompanying 
drawings, an embodiment of the present invention will 
be described. Fig. 1 shows an outline of an example of 
the structure of a data processing apparatus 1 
according to the present invention. As shown in Fig. 1, 
a picture data as input data is supplied from a picture 
processing device such as a television receiver, a 
video camera, a VTR, a scanner, or a digital camera to 
the data processing apparatus 1 according to the 
present invention . 

The data processing apparatus 1 according to 
the embodiment can execute process modes that 
accomplish for example the following contents 
corresponding to requests. Picture data that is 
generated as the processed result of each process mode 
is output as output data to an external device (for 
excimple, a displaying device or a recording and 
reproducing device) . 

(1) Increase of resolution (referred to as 
process mode ( 1 ) ) , 

(2) Generation of a picture dedicated for 
right eye and a picture dedicated for left eye 
(referred to as process mode (2)), 

(3) Generation of a luminance signal and 
color difference signals (referred to as process mode 
(3)), 



(4) Change of aspect ratio (referred to as 
process mode (4)), 

(5) Generation of a picture having a 
different resolution (referred to as process mode (5)), 
and 

(6) Conversion of frame rate (referred to as 
process mode (6)). 

The details of the processes of those process 
modes performed by the data processing apparatus 1 will 
be described later. 

The data processing apparatus 1 has a 
function controlling portion 11, a data processing 
portion 12, a pre-processing portion 13, and a post- 
processing portion 14. A command that causes a desired 
process to be executed is input to the function 
controlling portion 11. The function controlling 
portion 11 supplies a control signal corresponding to 
the input command to the data processing portion 12 so 
as to initialize the data processing portion 12 and 
executes a process corresponding to the command. In 
addition, the control signal that is output from the 
function controlling portion 11 is supplied to the pre- 
processing portion 13 and the post -processing portion 
14. The control signal causes the pre-processing 
portion IS and the post-processing portion 14 to 
execute predetermined processes. 

The pre-processing portion 13 performs a 



predetermined pre-process for picture data as input 
data corresponding to the control signal supplied from 
the function controlling portion 11 and supplies the 
processed data to the data processing portion 12. In 
5 addition, the control signal is supplied from the 

function controlling portion 11 to the data processing 
portion 12. The data processing portion 12 is 
initialized with the control signal- The data 
'~ processing portion 12 performs a classifying adaptive 

10 process corresponding to a process designated with the 

Q control signal for the supplied input data and outputs 

data generated as the processed result. An output of 
O the data processing portion 12 is supplied to the post- 

hj processing portion 14. The post-processing portion 14 

Tl5 performs a predetermined post -process for the output of 

the data processing portion 12 corresponding to the 
[^f control Signal supplied from the function controlling 

O portion 11 and outputs the result as output data to an 

external device . 
20 Fig. 2 shows an example of the structure of 

the pre-processing portion 13. The pre-processing 
portion 13 has selectors 130 and 132 and a plurality of 
delaying circuits 131A, 132B, and so forth. A control 
signal that is supplied from the function controlling 
25 portion 11 is input to the selectors 130 and 132 so as 

to control the selections of the signal paths of the 
selectors 130 and 132. The delaying circuits 131A, 



131B, and so forth have different delay times. For 
example, the delaying circuit 131A delays an input 
signal in the unit of a pixel. The delaying circuit 
131B delays an input signal in the unit of a line. The 
5 delaying circuit 13 IC delays an input signal in the 

unit of a frame. Signals that are output from the 
delaying circuits 131A, 131B, 131C, and so forth are 
input to the selector 132. The selector 132 selects 
the signals that are output from the delaying circuits 
10 131A, 131B, 131C, and so forth corresponding to the 

control signal supplied from the function controlling 
portion 11. The signals selected by the selector 132 
are output from the pre-processing portion 13. 
1="^ Alternatively, in the pre-processing portion 

C3L5 13, the signals that are output from the selector 130 

1^ may be directly input to the selector 132 not through 

\u the delaying circuits 131A, 131B, 131C, and so forth. 

J5 Further alternatively, the selector 130 may output the 

same signal in parallel as a plurality of outputs . 
20 Further alternatively, the pre-processing portion 13 

may have a processing circuit other than the delaying 
circuits 13 lA, 13 IB, and so forth. 

Fig. 3 shows an example of the structure of 
the data processing portion 12. The data processing 
25 portion 12 is composed of an input data processing 

portion 21, a selector 22, an intermediate data 
processing portion 23, an output data processing 
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portion 24, and a selector 25 • 

The input data processing portion 21 has a 
plurality of classifying adaptive processing portions 
(in this example, four classifying adaptive processing 
portions 31-1 to 31-4 (hereinafter simply referred to 
as classifying adaptive processing portion 31 when it 
is not necessary to distinguish them) (such notation 
applies to other structural portions) . The classifying 
adaptive processing portion 31 executes a classifying 
adaptive process for supplied data (namely, input data 
that is input to the data processing apparatus 1 or 
data that is input to the data processing apparatus 1 
and pre-processed by the pre-processing portion 13) and 
supplies the generated data as the processed result to 
the selector 22. 

The data that is processed (generated) in the 
input data processing portion 21 and the intermediate 
data processing portion 23 is supplied to the selector 
22. The select portion 22 selects a destination for 
the supplied data (the destination is for example a 
memory 41 of the intermediate data processing portion 
23 or a classifying adaptive processing portion 51 of 
the output data processing portion 24) and supplies the 
data to the selected destination. 

The intermediate data processing portion 23 
has a plurality of memories (in this example, four 
memories 41-1 to 41-4). The intermediate data 



processing portion 23 performs a write control and a 
read control of the memory 41 for data supplied from 
the selector 22. For example, the intermediate data 
processing portion 23 executes a process for changing a 
horizontal scanning sequence to a vertical scanning 
sequence. Data that is read from the memory 41 of the 
intermediate data processing portion 23 is returned to 
the selector 22. 

The output data processing portion 24 has a 
plurality of classifying adaptive processing portions 
(in this example, eight classifying adaptive processing 
portions 51-1 to 51-8). The classifying adaptive 
processing portion 51 executes a classifying adaptive 
process for the data supplied from the selector 22 
corresponding to the control signal received from the 
function controlling portion 11 and outputs the 
generated data as the processed result. Data that is 
output from the classifying adaptive processing 
portions 51-1 to 51-8 is input to the selector 25. The 
selector 25 selects data corresponding to the control 
signal that is supplied from the function controlling 
portion 11 and outputs the selected data. 

Fig. 4 shows an example of the structure of 
the post -processing portion 14. The post-processing 
portion 14 has a selector 140, a plurality of 
processing portions 141A, 141B, and so forth, and a 
selector 142. Data that is output from the data 



processing portion 12 is input to the selector 140. 
The selector 140 switches paths of the input data 
corresponding to a control signal that is supplied from 
the function controlling portion 11 and inputs the 
switched data to the plurality of the processing 
portions 141A, 141B, and so forth. 

According to the embodiment, the processing 
portion 141A is a line sequence converting portion that 
performs a conversion in the unit of a line. The 
processing portion 14 IB is a scanning direction 
converting portion that performs a conversion in the 
unit of a pixel. The processing portion 141C is a 
multiplexing portion that multiplexes data. The 
processing portions 141A, 141B, and so forth process 
input data corresponding to a control signal supplied 
from the function controlling portion 11. 

Next, the structure and operation of the data 
processing apparatus 1 that executes the processes for 
accomplishing the above -described process modes (1) to 
(6) will be described. 

Fig. 5 shows an example of the structure of 
the data processing apparatus 1 that executes the 
process for increasing the resolution in the process 
mode (1). In this example, an SD (Standard Density) 
picture (for example, picture data in an interlace 
format of which the number of lines is 525) 
(hereinafter, sometimes referred to as SD picture data) 
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is input as input data from an external device to the 
data processing apparatus 1. It is assumed that the 
data processing apparatus 1 generates an HD (High 
Density) picture (for example, picture data in a 
progressive format of which the number of lines is 525 
and of which the number of pixels in the horizontal 
direction is twice that of the SD picture) (hereinafter, 
sometimes , the HD picture data is referred to as HD 
picture data) . 

In this example, a command that causes the 
process for converting an SD picture into an HD picture 
to be executed is additionally input to the function 
controlling portion 11. The function controlling 
portion 11 outputs a control signal corresponding to 
the input command to the data processing portion 12 . 

The data processing portion 12 generates data 
of each line of the HD picture with the SD picture that 
is input as the input data to the data processing 
apparatus 1 corresponding to the control signal 
received from the function controlling portion 11* 
Accurately speaking, the data processing portion 12 
predicts pixels arranged on each line. 

In reality, as shown in Fig. 6, when a 
predetermined field of an SD picture is composed of 
lines LsD-i» LsD-2# Lsd-3# and so forth on which pixels 
denoted by large circles O (hereinafter referred to as 
SD pixels) are arranged, the data processing portion 12 



generates lines Lno-ir Lhd-2» Lhd-3 on which pixels of an 
HD picture denoted by small circles O are arranged at 
the same positions as the lines Lsd-i/ Lsd-2# LsD-3r and so 
forth of the SD picture (hereinafter, sometimes, a line 
of an HD picture at the same position as a line of an 
SD picture is referred to as line A) and lines Lhd-2/ 
L525P-4* and so forth of the HD picture at positions that 
are not same as the lines of the SD picture 
(hereinafter, a line of an HD picture at a position 
that is not same as a line of an SD picture is referred 
to as line B) . 

The data processing portion 12 separately 
generates pixels of line A and pixels of line B, In 
the post -processing portion 14 disposed downstream of 
the data processing portion 12, the selector 140 
selects the line sequence converting portion 14 lA and 
outputs line A and line B supplied from the data 
processing portion 12 in line sequence. In other words, 
HD picture data is output as output data from the line 
sequence converting portion 14 lA to an external device 
through the selector 142 . 

In other words, in this example, the data 
processing apparatus 1 non-interlaces an SD picture as 
an interlace format picture, doubles the number of 
pixels in the horizontal direction, and thereby 
generates an HD picture having pixels four times larger 
than an SD pixel. 

18 



Fig. 7 shows an example of the connections of 
the data processing portion 12 that executes the 
process for increasing the resolution in the process 
mode (1). In this example, in the input data 
processing portion 21, the two classifying adaptive 
processing portions 31-1 and 31-2 are used. SD picture 
data that is input to the data processing apparatus 1 
is supplied as input data to the classifying adaptive 
processing portions 31-1 and 31-2. 

The classifying adaptive processing portion 
31-1 executes a classifying adaptive process for 
generating line A (in the example shown in Fig. 6, Lhd-i# 
Lhd-3# Lhd-s, and so forth) that compose an HD picture 
with supplied SD picture data. In other words, in the 
classifying adaptive process, a process for predicting 
HD pixels arranged on line A is performed. The 
classifying adaptive processing portion 31-2 executes a 
classifying adaptive process for generating line B (in 
the example shown in Fig. 6, Lhd-2# LHD-4r and so forth) 
that compose an HD picture with an SD picture. In 
other words, in the classifying adaptive process, a 
process for predicting HD pixels arranged on line B is 
performed. 

Pixel data of line A and line B generated by 
the classifying adaptive processing portions 31-1 and 
31-2 is supplied to the select portion 22. 

The select portion 22 supplies pixel data of 
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line A and line B simultaneously supplied from the 
classifying adaptive processing portions 31-1 and 31-2 
of the input data processing portion 21 to the post- 
processing portion 14. In the post-processing portion 
14, the selector 140 selects the line sequence 
converting portion 141A corresponding to a control 
signal supplied from the function controlling portion 
11. Thus, line A and line B are supplied to the line 
sequence portion 14 lA. Line A and line B that compose 
the HD picture that are simultaneously supplied to the 
line sequence converting portion 13 are converted into 
a progressively scanned picture in the line sequence 
and output to an external device . 

The intermediate data processing portion 23 
and the output data processing portion 24 are not used 
in the process mode. 

Fig. 8 shows an example of the structure of 
the classifying adaptive processing portion 31-1 of the 
input data processing portion 21. As was described 
above, the classifying adaptive processing portion 31-1 
performs the process for line A. The classifying 
adaptive processing portion 31-1 comprises a 
classifying portion 61, an adaptive processing portion 
62, and a register group 63. The classifying portion 
61, the adaptive processing portion 62, and the 
register group 63 select a bank for the process mode 
corresponding to a control signal that is output from 




the function controlling portion 11 to the data 
processing portion 12 and execute various processes 
corresponding to a coefficient set of the selected bank. 

The classifying portion 61 is composed of a 
class tap extracting circuit 71 and a classifying 
circuit 72. Successively designating HD pixels on line 
A of an HD picture as considered pixels, the 
classifying portion 61 classifies the considered pixels 
as predetermined classes. 

In other words, the class tap extracting 
circuit 71 selects SD pixels (hereinafter, sometimes 
referred to as class taps) for each considered pixel 
from an SD picture, extracts the class taps, and 
supplies them to the classifying circuit 72. 

A register 63A of the register group 63 
registers information (hereinafter referred to as class 
tap extraction information) of a selected pattern of SD 
pixels as class taps for each classifying adaptive 
process executed by the classifying adaptive processing 
portion 31-1. In other words, before extracting class 
taps, the class tap extracting circuit 71 reads class 
tap extraction information necessary in the example 
(the class tap extraction information corresponds to a 
control signal supplied from the function controlling 
portion 11 to the data processing portion 12) and 
extracts class taps corresponding to the class tap 
extraction information . 



Fig. 9 shows an example of class tap 
extraction information in the process mode (1)- In Fig 
9, • (black circle) and O (white circle) represent SD 
pixels . Assuming that a considered pixel is a black 
circle, class taps are composed of 3 pixels x 3 pixels 
that are arranged around the considered pixel and that 
include the considered pixel. Predictive taps (that 
will be described later) are composed of 5 pixels x 5 
pixels wider than class taps by one pixel. In Fig.- 9, 
X marks represent HD pixels that are finally formed 
corresponding to a considered pixel. As represented in 
area A shown in Fig. 9, four HD pixels are formed with 
one considered pixel. As a result, above -described 
line B is formed. 

Next, with reference to Fig. 10, the process 
will be described in more detail. Fig. 10 is a 
schematic diagram showing the detail of one sequence 
containing a considered pixel in the vertical direction 
in Fig. 9. For example, SD pixels denoted by large 
circles are arranged in one sequence containing a 
considered pixel of an SD picture of a predetermined 
field (namely, in the vertical direction) . On the 
other hand, it is assumed that HD pixels denoted by 
small circles are arranged in one sequence 
corresponding to the above -described considered pixel 
of the SD picture. When HD pixel Yx of the SD picture 
is a considered pixel, corresponding to class tap 
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extraction information that is read from the register 
63A, a total of three SD pixels that are SD pixel X2 
that is the closest to HD pixel Yi (at the same 
position in the example shown in Fig. 10) and two SD 
pixels Xi and X3 vertically adjacent to SD pixel X2 are 
selected as class taps of the considered pixel (HD 
pixel) Yi. 

Returning to Fig. 8, the classifying circuit 
72 of the classifying portion 61 detects a feature of 
SD pixels that compose class taps supplied from the 
class tap extracting circuit 71 (for example, the 
distribution of pixel values) and supplies a class code 
that has been assigned for each feature to the adaptive 
processing portion 62 (coefficient memory 83). The 
class code is supplied as an address to the coefficient 
memory 83. As a pixel value, for example a luminance 
value of a pixel can be used. 

Since each pixel is assigned eight -bit data, 
in the example, assuming that each SD pixel is assigned 
eight-bit data, the number of classes becomes huge. 
Thus , the required memory capacity increases . 

Thus, actually, when classifying each 
considered pixel, the classifying circuit 72 performs 
the process for decreasing (compressing) the number of 
bits of each pixel that composes class taps. As an 
example of the method for the compressing process for 
the number of bits, ADRC (Adaptive Dynamic Range 



Coding) process is known. 

In the ADRC, the maximum pixel value MAX and 
the minimiam pixel value MIN are detected from pixels 
that compose a process block (class taps). In addition, 
5 the difference DR between the pixel value MAX and the 

pixel value MIN (= pixel value MAX - pixel value MIN) 
is calculated. The obtained DR is the dynamic range DR 
of the process block. The pixel value MIN is 
subtracted from each pixel value that composes the 
10 process block. Each subtracted value is divided by 

□ DR/2^. As a result, each pixel value that composes 

;S class taps as the process block is re-quantized to K 

P bits that are smaller than the originally assigned bits 

Li (eight bits). For example, when K = 1, in the example 

^"is shown in Fig. 10, the number of patterns of pixel 

f"' values of three SD pixels is (2^)^. In comparison with 

rU the case that the ADRC is not performed, the number of 

O patterns can be decreased. 

The compressing process of the classifying 
20 circuit 72 is not limited to the ADRC. Instead, 

another compressing process such as vector quantization 
may be used. 

The classifying circuit 72 decides the class 
of the considered pixel corresponding to the K-bit 
25 pixel value of each SD pixel that composes the class 

taps . 

Before classifying the considered pixel, the 
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classifying circuit 72 reads class tap extraction 
information necessary in the process mode from the 
register 63A of the register group 63, extracts the 
class taps corresponding to the class tap extraction 
information, and classifies the considered pixel with 
the extracted class taps . 

The adaptive processing portion 62 is 
composed of a predictive tap extracting circuit 81, a 
predictive calculating circuit 82., and a coefficient 
memory 83. The predictive tap extracting circuit 81 
selects SD pixels of the SD picture supplied to the 
adaptive processing portion 62 as predictive taps and 
supplies the predictive taps to the predictive 
calculating circuit 82. 

A register 63B of the register group 63 
registers information of selected patterns of 
predictive taps of the SD picture (hereinafter, 
sometimes, referred to as predictive tap extraction 
information) for each classifying adaptive process 
executed by the classifying adaptive processing portion 
31-1. In other words, before forming predictive taps, 
the predictive tap extracting circuit 81 reads required 
predictive tap extraction information (predictive tap 
extraction information corresponding to a control 
signal supplied from the function controlling portion 
11 to the data processing portion 12) from the register 
63B and extracts the predictive taps from the input 



data corresponding to the predictive tap extraction 
information. In the process mode (1), as was described 
with reference to Fig. 9, the predictive taps are 
composed of 5 pixels x 5 pixels arranged around the 
considered pixel. 

The coefficient memory 83 stores predictive 
coefficient sets (that are obtained in the learning 
process that will be described later) for individual 
classes. When a class is supplied from the classifying 
circuit 72 of the classifying portion 61, a predictive 
coefficient set is read from the address corresponding 
to the class and supplied to the predictive calculating 
circuit 82 . 

The coefficient memory 83 is composed of a 
plurality of banks. Each bank stores predictive 
coefficient sets corresponding to the individual 
process modes. Each bank stores predictive coefficient 
sets of corresponding types. A register 63C of the 
register group 63 registers bank selection information 
that is information of bank selections corresponding to 
process modes . 

Before reading a predictive coefficient set, 
the coefficient memory 83 reads bank selection 
information (that is coefficient information 
corresponding to a control signal supplied from the 
function controlling portion 11 to the data processing 
portion 12) corresponding to the process mode and sets 
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a bank corresponding to the bank selection information. 
As described above, in the example, since the 
classifying adaptive processing portion 31-1 performs 
the process for line A, the coefficient memory 83 sets 
a bank that stores a predictive coefficient set 
corresponding to line A. Predictive coefficients are 
read from an address of the bank corresponding to the 
supplied class and supplied to the predictive 
calculating circuit _82. 

Predictive taps are supplied from the 
predictive tap extracting circuit 81 to the predictive 
calculating circuit 82. In addition, a predictive 
coefficient set is supplied from the coefficient memory 
83 to the predictive calculating circuit 82. 

The predictive calculating circuit 82 
performs a calculation of Expression (1) that is a 
linear combination model defined by a linear 
combination of a predictive coefficient W and a pixel 
value X using a predictive coefficient set (predictive 
coefficients wi, W2, and so forth) supplied from the 
coefficient memory 83 and pixel values (pixel values Xi, 
X2, and so forth) of pixels that compose predictive 
taps supplied from the coefficient memory 83, obtains a 
predictive value E [y] of the considered pixel (HD 
pixel) y, and treats the calculated result as the pixel 
value of a HD pixel (HD pixel arranged on line A) . 

E [y] = WiXi + W2X2 + ... ... (1) 



Alternatively, using a non-linear combination 
model, a calculation can be performed. 

As described above, pixel values of 
individual HD pixels arranged on line A of an HD 
picture are predicted. Thus, line A is generated. 

The structure of which the classifying 
adaptive processing portion 31-2 performs a predictive 
process as a classifying adaptive process in the 
process mode (1) is basically the same as the structure 
of the above -described classifying adaptive processing 
portion 31-1. In other words, in the classifying 
adaptive processing portion 31-2, the class tap 
extracting circuit 71 and the coefficient memory 83 of 
the classifying portion 61 and the adaptive processing 
portion 62 extract class taps, predictive taps, and a 
predictive coefficient set corresponding to required 
tap extraction information, predictive tap extraction 
information, and bank selection information. As a 
result, pixel values of HD pixels arranged on line B of 
an HD picture (namely, a line at a position that is not 
the same as a line of the SD picture) are predicted. 
For example, the pixel value of HD pixel Y2 shown in 
Fig. 10 is predicted. As a result, line B is generated. 

Fig. 11 shows an example of the structure of 
a learning device that performs a learning process for 
calculating predictive coefficients that are pre- stored 
to the coefficient memory 83 shown in Fig. 8. In the 
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process mode (1), the learning process is performed for 
line A and line B. The learnt results for line A and 
line B are separately output . 

An HD picture as teacher data in the learning 
5 process is supplied to both a thin-out circuit 91 and a 

teacher data extracting circuit 95. The thin-out 
circuit 91 thins out pixels of the HD picture as the 
teacher data so as to generate an SD picture. The 
thin-out circuit 91 supplies the SD picture to a 
10 classifying portion 92 and a predictive tap extracting 

circuit 93. For example, the thin -out circuit 91 
halves the number of pixels in each of the horizontal 

»^ 

ffl direction and the vertical direction of the HD picture 

so as to form an SD picture. 

1315 The classifying portion 92 decides the class 

of the considered pixel in the same process as the 

m classifying portion 61 shown in Fig. 8 and supplies the 

'ii decided class to address terminals (AD) of a predictive 

tap memojry 94 and a teacher data memory 96. The 
20 predictive tap extracting circuit 93 performs the same 

process as the predictive tap extracting circuit 81 
shown in Fig. 8 so as to extract predictive taps. The 
predictive tap extraction circuit 93 supplies the 
extracted predictive taps to the predictive tap memory 
25 94. 

The predictive tap memory 94 stores the 
predictive taps supplied from the predictive tap 
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extracting circuit 93 at an address corresponding to 
the class supplied from the classifying portion 92. 

On the other hand, the teacher data 
extracting circuit 95 extracts an HD pixel as a 
5 considered pixel (designated by the classifying portion 

92 and the predictive tap extracting circuit 93) from 
the supplied HD picture and supplies the HD pixel as 
teacher data to the teacher data memory 96. 

The teacher data memory 96 stores teacher 
10 data supplied from the teacher data extracting circuit 

3 95 at an address corresponding to the class supplied 

y from the classifying portion 92. 

fj The above-described process is performed for 

y all pixels as considered pixels that compose an HD 

^15 picture prepared for the learning process. 

" As a result, at the same address of the 

y teacher data memory 96 or the predictive tap memory 94, 

3 an HD pixel of the class corresponding to the address 

as teacher data and SD pixels at the positions of 
20 predictive taps of the HD pixels as learnt data are 

stored. 

In the predictive tap memory 94 and the 
teacher data memory 96, a plurality of pieces of 
information can be stored at the same address. Thus, a 
25 plurality of pieces of learnt data and teacher data 

that are classified as the same class can be stored at 
the same address. 
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A calculating circuit 97 reads predictive 



taps as learnt data or HD pixels as teacher data stored 
at the seime address of the predictive tap memory 94 or 
the teacher data memory 96, respectively. Using the 
predictive taps or HD pixels that are read, by for 
example the method of least squares, a predictive 
coefficient of which the difference between predictive 
value and teacher data becomes minimum is obtained. In 
other words, the calculating circuit 97 solves a normal 
equation as Expression (2) for each class so as to 
obtain a predictive coefficient. 



The predictive coefficient set that has been 
obtained in the above -described manner is stored to the 
coefficient memory 83 shown in Fig. 8. 

Since the structure of the classifying 
adaptive processing portion 31-2 of the input data 
processing portion 21 that performs the learning 
process is the same as that of the above -described 
classifying adaptive processing portion 31-1, the 
description will be omitted. 

In the above- described example, luminance 
values are used as pixel values. When a signal is 





(2) 



composed of luminance and color differences, the above- 
described process is performed for each of the 
luminance and color differences so as to generate an HD 
picture. In the case, the classifying adaptive 
5 processing portions 31-3 and 31-4 of the input data 

processing portion 21 may execute a classifying 
adaptive process using color differences and generate 
line A and line B that compose an HD picture. 

The classifying adaptive process for 
10 converting an interlaced picture into a non-interlaced 

3 picture is described in Japanese Patent Application No. 

P HEX 10-208116 (Japanese Patent Laid-Open Publication No, 

3 2000-41223 laid open on February 8, 2000) and the 

y corresponding United State Patent Application (serial 

is number 09/358272, filed on July 21, 1999) both of which 

2 were filed by the applicant of the present invention, 
y Although the normal equation as Expression 

3 (2) used in obtaining predictive coefficients has been 
disclosed in prior art (for example, Japanese Patent 

20 Laid-open Publication No. 2000-41223 laid open on 

February 8, 2000), for allowing the reader to easily 
understand the present application, it will be 
described in detail. 

To generalize Expression (1) of the linear 

25 combination model defined by linear combination of 

predictive coefficients and pixel values for 
calculating a predictive value E, defining a matrix W 
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as a set of predictive coefficients Wj, a matrix X as a 
set of student data Xij, and a matrix Y' as a set of 
predictive values E [yi] as Expression (3), an 
observation equation as Expression (4) is satisfied. 



X = 



^1/ 

^2y 



. . . (3) 



1.4.0 
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E[yj] 



w ^ 



xw = y • . • (4) 

In this case, a component Xi-j of the matrix X 
represents j-th student data of an i-th set of student 
data (a set of student data used for predicting i-th 
teacher data yi) • A component Wj of the matrix W 
represents a predictive coefficient that is multiplied 
by j-th student data of a set of the student data, yi 
represents i-th teacher data. Thus, E [ytl represents 
a predictive value of 1-th teacher data. In other 
words, y on the left side of Expression (1) is obtained 
by omitting the suffix 1 of the component yi of the 
matrix Y. xi, X2, ... of the right side of Expression 
(1) are obtained by omitting the suffix 1 of the 
component Xij of the matrix X. 
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Now, by applying the method of least squares 
to the observation equation, a predictive value E [y] 
close to a pixel value y of an HD picture is obtained. 
In this case, when the matrix E as a set of residuals e 
of a predictive value E [y] to a pixel value y of an HD 
pixel is defined with Expression (5), a residual 
formula as Expression (6) is satisfied with Expression 
(4)- 









^2 

















(5) 



XW = Y + E . . • ( 6 ) 

In this case, the predictive coefficient Wi 
for obtaining the predictive value E [y] close to a 
pixel value y of an HD picture is obtained by 
minimizing the square error of Expression (7). 



I' 



(7) 



Thus, when the result of which the square 
error is differentiated with the predictive coefficient 
Wj becomes 0, (in other words) the predictive 
coefficient Wj that satisfies Expression (8) becomes an 
optimum value for obtaining the predictive value E [y] 
that is close to the pixel value y of an HD pixel. 



de 



2 +... + e,^ = 0O=l,2,...,J) 



(8) 



34 



Thus, when Expression (6) is differentiated 
with the predictive coefficient Wj, Expression (9) is 
satisfied. 

TT^^il-^^Jr ^^''ZVJ ' = =1,2,...,/) ... (9) 



5 With Expression (8) and Expression (9), 

Expression (10) is obtained. 

2^/^n=0,2^i^i2=0v-2^'^^^^^ ... (10) 

Considering the relation among the student 
data Xij, predictive coefficients Wj, teacher data yi, 
%.0 and residuals ej of the residual equation as Expression 

52 (6), with Expression (10), the normal equations of 

H Expression (2) can be obtained. In this case, for each 

f5 normal equation as Expression (2), it is preferred to 

prepare a predetermined number of sets of student data 
njLS xij and teacher data yi. Thus, the normal equations 

53 corresponding to the number J of predictive 

"^^^ coefficients Wj can be obtained. By solving Expression 

(2), an optimum predictive coefficient w^ can be 
obtained. 

20 Fig. 12 shows an example of the structure of 

the data processing apparatus 1 that performs another 
processing method for increasing the resolution in the 
process mode (1). In the other processing method for 
the process mode (1), the post-processing portion 14 

25 selects a scanning direction converting portion 14B. 
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The function controlling portion 11 outputs a 
control signal corresponding to a conunand that is input 
in this case to the data processing portion 12. 

The data processing portion 12 is initialized 
with a control signal supplied from the function 
controlling portion 11. Thereafter, the number of 
pixels in the horizontal direction of an SD picture as 
input data supplied from an external device is doubled. 
Thereafter, a horizontal scanning sequence (television 
raster scanning sequence) is converted into a vertical 
scanning sequence. Moreover, the number of pixels in 
the vertical direction is doubled. In other words, in 
the case, an HD picture with a higher resolution than 
an SD picture is generated. 

However, since picture data that is output 
from the data processing portion 12 is still in the 
vertical scanning sequence, it is necessary to restore 
the picture data to the horizontal scanning sequence. 
Thus, in the data processing apparatus 1, the post- 
processing portion 14 disposed downstream of the data 
processing portion 12 selects the scanning direction 
converting portion 141B corresponding to a control 
signal supplied from the function controlling portion 
11. 

As with the intermediate data processing 
portion 23 of the data processing portion 12, the 
scanning direction converting portion 14 IB has a memory. 
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By controlling writing and/or reading process of the 
picture data that is supplied from the data processing 
portion 12 to/from the memory, the vertical scanning 
sequence is converted into the horizontal scanning 
sequence. For example, picture data that is supplied 
from the data processing portion 12 to the scanning 
direction converting portion 141B is address -controlled 
so that pixels are written in the vertical scanning 
direction to the memory. VHien picture data is read 
from the memory, the memory is address -controlled so 
that pixels are read in the horizontal scanning 
direction. Thus, the scanning direction of data is 
converted from the vertical scanning direction to the 
horizontal scanning direction. The picture data 
restored to the horizontal scanning sequence by the 
scanning direction converting portion 141B is output as 
output data to an external device. 

Fig. 13 shows an example of the connections 
of the data processing portion 12 that performs the 
other processing method in the process mode (1). In 
this example, in the input data processing portion 21, 
one classifying adaptive processing portion 31-1 is 
used. SD picture data that is input as input data to 
the data processing apparatus 1 is supplied to the 
classifying adaptive processing portion 31-1. 

The classifying adaptive processing portion 
31-1 performs a classifying adaptive process for 



generating a picture whose number of pixels in the 
horizontal direction is twice as many as the supplied 
SD picture data (hereinafter, sometimes, the generated 
picture is referred to as intermediate picture) and 
supplies picture data of the intermediate picture 
(hereinafter, sometimes, the picture data is referred 
to as intermediate picture data) to the selector 22. 

In the example, in the intermediate data 
processing portion 23, the memory 41-1 is used. The 
intermediate picture data is supplied from the select 
portion 22 to the memory 41-1. The intermediate data 
processing portion 23 address -controls the memory 41-1 
so that the data written in the horizontal scanning 
direction to the memory 41-1 is read in the vertical 
scanning direction therefrom. Thus, the intermediate 
data processing portion 23 converts the horizontal 
scanning sequence of the intermediate picture into the 
vertical scanning sequence and returns the converted 
picture to the selector 22. 

In this example, in the output data 
processing portion 24, the classifying adaptive 
processing portion 51-1 is used. The intermediate 
picture data is supplied from the selector 22 to the 
classifying adaptive processing portion 51-1. The 
classifying adaptive processing portion 51-1 performs a 
classifying adaptive process for doubling the number of 
pixels in the vertical direction of the intermediate 
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picture corresponding to the intermediate picture data 
supplied from the select portion 22 and supplies the 
generated picture data to the post -processing portion 
14 disposed downstream of the classifying adaptive 
processing portion 51-1. 

The picture data supplied to the post- 
processing portion 14 is supplied to the scanning 
direction converting portion 141B selected by the 
selector 140 corresponding to a control signal supplied 
from the function controlling portion 11. The scanning 
direction converting portion 141B performs a write 
control and a read control for the memory so that 
picture data that is written in the vertical scanning 
direction to the memory is read in the horizontal 
scanning direction therefrom. As a result, the 
vertical scanning sequence of the picture data is 
restored to the horizontal scanning sequence. The 
picture (HD picture) restored to the horizontal 
scanning sequence by the scanning direction converting 
portion 14 IB is output as output data to an external 
device . 

Next, the predictive process of the 
classifying adaptive processing portion 31-1 for the 
input data processing portion 21 according to the other 
processing method in the process mode (1) will be 
described. 

The class tap extracting circuit 71 (see Fig. 



8) of the classifying adaptive processing portion 31-1 
reads class tap extraction information required in this 
example from the register 63A of the register group 63, 
forms class taps corresponding to the class tap 
extraction information, and supplies the class taps to 
the classifying circuit 72. 

Fig- 14 shows an example of class tap 
extraction information corresponding to the other 
processing method in the process mode (1). Fig. 14 
shows an example of class tap extraction information in 
the process for doubling the number of pixels in the 
horizontal scanning direction in the process mode (1). 
In Fig. 14, # (black circle) and O (white circle) 
represent SD pixels. Assuming that a considered pixel 
is a black circle pixel, class taps are composed of 
seven pixels arranged on a line of which the considered 
pixel is placed at the center thereof. On the other 
hand, predictive taps that will be described later are 
composed of 11 pixels that are wider than the class 
taps by two pixels. 

The class taps and predictive taps used for 
the process for doubling the number of pixels in the 
vertical direction may be arranged in the vertical 
direction rather than the horizontal direction shown in 
Fig. 14. 

Next, with reference to Fig. 15, this process 
will be described in detail. It is assumed that lines 
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K-1 to K+1 are arranged in a predetermined field of an 
SD picture as input data. In Fig. 15, large circles o 
represent SD pixels . Small circles represent pixels of 
an HD picture whose number of pixels in the horizontal 
direction is twice as many as an SD picture. The 
pixels are arranged as input data on lines K-1 to K+1 
of a predetermined field of the SD picture as shown in 
Fig. 15. 

Now, the case that pixel Yk,5 of an 
intermediate picture on line K is a considered pixel is 
considered. In the example, according to the class tap 
extraction information that is read from the register 
group 6 3A, a total of seven SD pixels that are SD pixel 
Xk.3 that is predicted as a pixel having the highest 
correlation to pixel Yk,5 on line K, two SD pixels Xk.2 
and Xk.i arranged on the left of SD pixel Xk,3, two SD 
pixels Xk,4 and XK,5 arranged on the right of SD pixel 
Xk,3, and SD pixels Xk.1,3 and SD pixel Xk+i.3 arranged on 
line K-1 and line K+1 at the positions corresponding to 
SD pixel Xk,3 are selected as class taps. 

Before classifying the considered pixel, the 
classifying circuit 72 reads class tap extraction 
information required in the process mode from the 
register 63A of the register group 63, extracts class 
taps corresponding to the class tap extraction 
information, and classifies the considered pixel using 
the extracted class taps. The decided class of the 



considered pixel is supplied as an address to the 
coefficient memory 83 of the adaptive processing 
portion 62. 

The predictive tap extracting circuit 81 of 
the adaptive processing portion 62 reads predictive tap 
extraction information required in the example from the 
register 63B of the register group 63, forms predictive 
taps corresponding to the predictive tap extraction 
information, and supplies the predictive taps to the 
predictive calculating circuit 82- 

The coefficient memory 83 reads coefficient 
information required in the example from the register 
63C of the register group 63. In the coefficient 
memory 83, a bank used corresponding to the predictive 
information is set. From the bank, a predictive 
coefficient set for doubling the number of pixels in 
the horizontal direction is read from predictive 
coefficient sets stored at an address corresponding to 
the class supplied from the classifying circuit 72. 
The predictive coefficient set that is read from the 
bank is supplied to the predictive calculating circuit 
82. 

The predictive calculating circuit 82 
calculates Expression (1) with the predictive 
coefficient set supplied from the coefficient memory 83 
and pixel values of SD pixels that compose predictive 
taps supplied from the predictive tap extracting 
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circuit 81 and treats the predictive value of the 
considered pixel as a pixel value of a pixel of the 
intermediate picture. The intermediate picture 
composed of the predicted pixels is supplied to the 
5 selector 22. 

Since the learning process according to the 
other processing method in the process mode (1) is 
basically the same as that of the data processing 
apparatus 1 in the process mode (1), the description 
10 thereof will be omitted. 

Since the predictive process of the 
classifying adaptive processing portion 51-1 of the 
ffi output data processing portion 24 in the example of the 

1=^ other processing method in the process mode (1) is 

1315 basically the same as that of the classifying adaptive 

1^ processing portion 31-1 of he input data processing 

portion 21 of the process mode (1), the detailed 
description thereof will be omitted. The classifying 
portion 61 of the classifying adaptive processing 
20 portion 51-1 and the predictive tap extracting circuit 

81 and the coefficient memory 83 of the adaptive 
processing portion 6 2 extract class taps and predictive 
taps corresponding to tap extraction information, 
predictive tap information, and coefficient information 
25 required in the example of the other processing method 

in the process mode (1) and read a predictive 
coefficient set from the coefficient memory 83. 
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Corresponding to the results, pixels for doubling the 
number of pixels in the vertical direction are 
predicted. 

In addition, since the learning process of 
the classifying adaptive processing portion 51-1 of the 
output data processing portion 24 according to the 
other processing method in the process mode (1) is 
basically the same as the learning process of the 
classifying adaptive processing portion 31-1 in the 
above -described process mode (1), the description 
thereof will be omitted. 

In the above description, the classifying 
adaptive processing portion 51-1 of the output data 
processing portion 24 executes the classifying adaptive 
process for generating a picture whose number of pixels 
in the vertical direction is doubled so as to generate 
an HD picture. However, the present invention is not 
limited to such an example. For example, as with the 
above -described process mode (1), according to the 
other processing method in the process mode (1), a 
classifying adaptive process for predicting pixels 
arranged on a line (line A) of which a line of an SD 
picture is present at the same positions as pixels that 
compose an HD picture and another classifying adaptive 
process for predicting pixels arranged on a line (line 
B) of which a line of an SD picture is not present at 
the same positions as pixels that compose an HD picture 




can be Independently executed. In this case, these 
classifying adaptive processes of two classifying 
adaptive processing portions (for example, the 
classifying adaptive processing portions 51-1 and 51-2) 
5 of the output data processing portion 24 can be 

executed. 

In the above -described classifying adaptive 
process is described in International Patent Laid-Open 
No. WO 96/07987 and the corresponding United State 
10 Patent No. 5,903,481 filed by the applicant of the 

^ present invention. 

2 Fig. 16 shows an example of the structure of 

y the data processing apparatus 1 that executes a process 

for generating a picture dedicated for a right eye and 

3.5 a picture dedicated for a left eye in the above- 

described process mode (2). In the process mode (2), 

U in the post -processing portion 14, the multiplexing 

2 portion 141C is used. 

In the example, the data processing apparatus 

20 1 generates a picture dedicated for a left eye 

(hereinafter referred to as left -eye picture) and a 
picture dedicated for a right eye (hereinafter referred 
to as right-eye picture) with SD picture data as input 
data supplied from an external device and outputs 

25 picture data (hereinafter referred to as stereo picture 

data) of the multiplexed picture (hereinafter referred 
to as stereo picture) as output data to an external 
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device . 

In this case, it is assumed that the SD 
picture data as input data is a television broadcast 
signal that has been A/D converted with a sampling 
clock of 13.5 MHz. In other words, the size of the SD 
picture is around 720 pixels (horizontal) x 480 lines 
(vertical) per frame. 

In this example, a command that causes the 
process for generating a left -eye picture and a right - 
eye picture to be executed is input to the function 
controlling portion 11. The function controlling 
portion 11 outputs a control signal corresponding to 
the input command to the data processing portion 12. 

The function controlling portion 11 also 
extracts a vertical synchronous signal and a horizontal 
synchronous signal from the television broadcast signal 
and controls the data processing portion 12 and the 
multiplexing portion 141C of the post-processing 
portion 14 corresponding to the timings thereof. 

After the data processing portion 12 is 
initialized with a control signal supplied from the 
function controlling portion 11, the data processing 
portion 12 generates a left -eye picture and a right -eye 
picture with an SD picture that is input as input data. 

The multiplexing portion 141C of the post- 
processing portion 14 disposed downstream of the data 
processing portion 12 multiplexes the left -eye picture 
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data and the right -eye picture data supplied from the 
data processing portion 12, generates stereo picture 
data, and outputs the generated stereo picture data as 
output data to an external device. 
5 Fig. 17 shows an example of the connections 

of the data processing portion 12 in the process mode 
(2). In the example, in the input data processing 
portion 21, the two classifying adaptive processing 
portions 31-1 and 31-2 are used. SD picture data that 
10 is input as input data to the data processing apparatus 

... 1 is supplied to the classifying adaptive processing 

portions 31-1 and 31-2. 
0 The classifying adaptive processing portion 

- 31-1 performs a classifying adaptive process for 

3L5 generating left -eye picture data with the supplied SD 

picture data. The classifying adaptive processing 
portion 31-1 supplies the generated left -eye picture 
^ data to the selector 22. On the other hand, the 

classifying adaptive processing portion 31-2 performs a 
20 classifying adaptive process for generating right -eye 

picture data with the supplied SD picture data. The 
classifying adaptive processing portion 31-2 supplies 
the generated right-eye picture data to the selector 22 
The selector 22 supplies the left -eye picture 
25 data and the right-eye picture data supplied from the 

classifying adaptive processing portions 31-1 and 31-2 
of the input data processing portion 21 to the post- 
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processing portion 14. In the post-processing portion 
14, the route of the selector 140 for the supplied 
left -eye picture data and right -eye picture data is 
controlled corresponding to a control signal supplied 
from the function controlling portion 11, The left -eye 
picture data and right -eye picture data are supplied to 
the multiplexing portion 141C. 

In the process mode (2), the intermediate 
data processing portion 23 and the output data 
processing portion 24 are not used. 

Next, the predictive process of the 
classifying adaptive processing portion 31-1 of the 
input data processing portion 21 in the exeimple will be 
described. 

The class tap extracting circuit 71 (see Fig. 
8) of the classifying adaptive processing portion 31-1 
reads class tap extraction information required in the 
example from the register 63A of the register group 63. 
The classifying adaptive processing portion 31-1 
designates each of pixels that compose the left -eye 
picture finally obtained in the classifying adaptive 
process as a considered pixel. The left -eye picture 
does not actually exist before the predictive process 
is performed. The classifying adaptive processing 
portion 31-1 extracts class taps corresponding to class 
tap extraction information corresponding to each 
considered pixel and supplies the extracted class taps 
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to the classifying circuit 72. 

Figs. 18A and 18B show examples of class tap 
extraction information in the process mode (2). Fig. 
18A shows class tap extraction information for a left- 
5 eye picture. Fig. 18B shows class tap extraction 

information for a right-eye picture. In Figs. 18A and 
18B, # (black circle) and O (white circle) represent 
SD pixels. Now, it is assumed that in a left-eye 
picture shown in Fig. 18A, a considered pixel is a 
10 black circle. In this case, class taps are composed of 

f3 4 pixels X 3 pixels that contains a considered pixel 

% and that slightly deviate from the considered pixel (in 

jy the example, by 0.5 pixels). On the other hand, 

predictive taps (that will be described later) are 
■^3l5 composed of 7 pixels x 3 pixels that are wider to the 

right than the class taps by one pixel and to the left 
fU than the class taps by two pixels. 

□ Unlike the left -side picture, in the right- 

side picture shown in Fig. 18B, class taps are composed 
20 of 4 pixels x 3 pixels that slightly deviate from a 

considered pixel (by 0.5 pixels as with the left-eye 
picture). In the right-side picture, predictive taps 
are composed of 7 pixels x 3 pixels that are wider to 
the left of the class taps by one pixel and to the 
25 right of the class taps by two pixels. 

The classifying circuit 72 reads class tap 
extraction information required in the process mode (2) 
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from the register 63A of the register group 63, decides 
the class of the considered pixel corresponding to the 
class tap extraction information, and supplies the 
class to the coefficient memory 83 of the adaptive 
5 processing portion 62, 

The predictive tap extracting circuit 81 of 
the adaptive processing portion 62 reads predictive tap 
extraction information required in the process mode (2) 
from the register 63B of the register group 63. The 
10 predictive tap extracting circuit 81 extracts 

i predictive taps from an SD picture supplied to the 

3 adaptive processing portion 62 for the considered pixel 

3 corresponding to the predictive tap extraction 

J information that has been read and supplies the 

is extracted predictive taps to the predictive calculating 

^ circuit 82. For example, as practically shown in Fig. 

S 19, predictive taps ARl denoted by a dotted line are 

f formed and supplied to the predictive calculating 

circuit 82. 

20 The coefficient memory 83 reads coefficient 

information required in the example from the register 
63C of the register group 63 and sets a bank that is 
used corresponding to the coefficient information. The 
coefficient memory 83 reads a predictive coefficient 

25 set for generating a left -eye picture from predictive 

coefficient sets stored at an address of the bank 
corresponding to the class supplied from the 



# m 

classifying circuit 72 and supplies the predictive 
coefficient set to the predictive calculating circuit 
82. 

The predictive calculating circuit 82 
calculates Expression (1) with the predictive 
coefficient set supplied from the coefficient memory 83 
and pixel values of the SD picture that compose the 
predictive taps supplied from the predictive tap 
coefficient circuit 81 and designates the predictive 
value of the considered pixel as the pixel value of a 
pixel that composes the left-eye picture. 

The left-eye picture composed of pixels 
having pixel values that have been predicted in such a 
manner are supplied to the selector 22. 

Likewise, the classifying adaptive processing 
portion 31-2 processes a right-eye picture. Since the 
predictive process of the classifying adaptive 
processing portion 31-2 is basically the same as the 
predictive process of the classifying adaptive 
processing portion 31-1, the detailed description 
thereof will be omitted. The classifying portion 61 of 
the classifying adaprive processing portion 31-2, the 
predictive tap extracting circuit 81 of the adaptive 
processing portion 62, and the coefficient memory 83 
extract class taps corresponding to tap extraction 
information, predictive tap extraction information, and 
coefficient information required in the process mode 



(2) and read a predictive coefficient set from the 
coefficient memory 83. 

For example, the predictive tap extracting 
circuit 81 forms predictive taps AR2 (denoted by a dot- 
5 dash line shown in Fig. 19) that deviate by several 

pixels to the right of the predictive taps ARl formed 
by the classifying adaptive processing portion 31-1. 
The amount of deviation between the predictive taps ARl 
and AR2 can be set corresponding to parallax designated 
10 between a left -eye picture and a right -eye picture that 

are generated. 

^-S In addition, the coefficient memory 83 of the 

C3 classifying adaptive processing portion 31-2 reads a 

hi predictive coefficient set for generating a right -eye 

^15 picture from the bank and supplies the predictive 

L coefficient set to the predictive calculating circuit 

fl 82. The predictive calculating circuit 82 calculates 

Expression (1) with the predictive coefficient set 
supplied from the coefficient memory 83 and the 
20 predictive taps supplied from the predictive tap 

extracting circuit 81 and designates the predictive 
value of the considered pixel as the pixel value of a 
pixel that composes the right-eye picture. 

The right -eye picture composed of pixels 
25 having predicted pixel values is supplied to the 

selector 22. 

Next, the learning process in the process 
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mode (2) will be described. When the learning process 
is performed by the classifying adaptive processing 
portion 31-1, as shown in Fig. 20, an object is 
photographed by three television cameras (a left -eye 
Ccunera L, a right -eye camera R, and a center camera C) 
disposed at positions having respective parallaxes. 
Among the three photographed pictures, pictures 
photographed by the left -eye camera L and the center 
camera C are used. However, in the learning process of 
the classifying adaptive processing portion 31-2, 
pictures photographed by the right -eye camera R and the 
center camera C are used. 

Fig. 21 shows an example of the structure of 
the classifying adaptive processing portions 31-1 and 
31-2 that perform a learning process in the process 
mode (2). In the structure, the thin-out circuit 91 of 
the structure shown in Fig. 11 is not used. 

A picture photographed by the center camera C 
is supplied to the classifying portion 92 and the 
predictive tap extracting circuit 93. A picture 
photographed by the left -eye camera L is supplied to 
the teacher data extracting circuit 95. In other words, 
in the example, the picture photographed by the center 
camera C is classified (namely, a class of the picture 
is decided). With teacher data that is a picture 
photographed by the left -eye camera L, a predictive 
coefficient that represents the correlation between the 



picture photographed by the center camera C and the 
picture photographed by the left -eye camera L is 
obtained for each class. 

A set of predictive coefficients that have 
been obtained in such a manner is stored to the 
coefficient memory 83 of the classifying adaptive 
processing portion 31-1. 

Since the processes of the classifying 
portion 92 to the calculating circuit 97 are basically 
the same as those in the case shown in Fig. 11, the 
description thereof will be omitted. 

The learning process of the classifying 
adaptive processing portion 31-2 is the Scime as that of 
the classifying adaptive processing portion 31-1 except 
that a picture photographed by the right -eye camera R 
is teacher data. In other words, a predictive 
coefficient that represents the correlation between the 
picture photographed by the center camera C and the 
picture photographed by the right -eye camera R is 
obtained. A set of predictive coefficients that have 
been obtained in such a manner is stored to the 
coefficient memory 83 of the classifying adaptive 
processing portion 31-2. 

The classifying adaptive process for 
generating a left-eye picture and a right-eye picture 
is described in Japanese Patent Laid-open Publication 
No. HEX 9-55962 (laid open on February 25, 1997) that 
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was filed by the applicant of the present invention. 

Fig. 22 shows an example of the structure the 
data processing apparatus 1 that executes a process for 
generating a luminance signal and color difference 
signals in the process mode (3). In this example, an 
NTSC (National Television System Committee) format 
picture (hereinafter referred to as NTSC picture) is 
supplied from an external device to the data processing 
apparatus 1. In this example, it is assumed that the 
NTSC picture data supplied to the data processing 
apparatus 1 is only an SD picture as a so-called 
composite video signal of which a Iximinance signal Y 
and color difference signals U and V are frequency- 
multiplexed (hereinafter, the composite video signal is 
referred to as composite signal NTSC) . 

The data processing apparatus 1 separates the 
supplied NTSC picture data into luminance Y and color 
differences U and V. When the NTSC picture data is 
supplied to the data processing apparatus 1, the pre- 
proceissing portion 13 performs a gain compensating 
process, a subcarrier reproducing process, a phase 
shifting process, and so forth for the NTSC picture 
data and supplies the processed NTSC picture data to 
the data processing portion 12. The data processing 
portion 12 performs the classifying adaptive process 
for the supplied data corresponding to the pre-learnt 
predictive coefficient set, generates the luminance 
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signal Y and the color difference signals U and V, and 
outputs these signals. Hereinafter, these output 
signals are together referred to as component signal (Y, 
U, V). In the process mode (3), the pre-processing 
5 portion 13 is not the above -described delaying circuit, 

but a circuit that performs the gain compensating 
process, sub-carrier reproducing process, phase 
shifting process, and so forth. 

Fig. 23 shows an excimple of the connections 
10 of the data processing portion 12 in the process mode 

W (3). In the process mode (3), in the input data 

processing portion 21, the three classifying adaptive 

O processing portions 31-1, 31-2, and 31-3 are used. 

ly Signals NTSC-Y, NTSC-U, and NTSC-V of which the SD 

35 picture data that is input as input data to the data 

processing apparatus 1 is pre-processed in a 
predetermined manner (that will be described later) are 
supplied to the classifying adaptive processing 
portions 31-1, 31-2, and 31-3, respectively. 
20 The classifying adaptive processing portion 

31-1 performs a classifying adaptive process for 
generating the luminance signal Y with the supplied 
signal NTSC-Y and supplies the generated luminance 
signal Y to the selector 22. Likewise, the classifying 
25 adaptive processing portions 31-2 and 31-3 perform 

classifying adaptive processes for generating the color 
difference signals U and V with the supplied signals 



NTSC-U and NTSC-V, respectively, and supply the 
generated color difference signals U and V to the 
selector 22. 

The selector 22 supplies the luminance signal 
Y and the color difference signals U and V supplied 
from the classifying adaptive processing portions 31-1, 
31-2, and 31-3 of the input data processing portion 21 
as output signals to the post-processing portion 14, 
The post -processing portion 14 outputs the supplied 
signals as they are. 

In the process mode (2), the intermediate 
data processing portion 23 and the output data 
processing portion 24 are not used. 

Fig. 24 shows an example of the structure of 
the data processing apparatus 1 that accomplishes a Y/C 
separating device that performs a classifying adaptive 
process for separating the composite signal NTSC into 
the component signal (Y, U, V) in the process mode (3). 
As denoted by a dot line shown in Fig. 24, a block 
disposed upstream of classifying circuits 50, 51, and 
52 corresponds to the pre-processing portion 13. A 
block disposed downstrecim of the classifying circuits 
50, 51, and 52 corresponds to the classifying adaptive 
processing portions 31-1, 31-2, and 31-3. A 
classifying portion 50 corresponds to the classifying 
portion 61 of the classifying adaptive processing 
portion 31. A predictive filter 53 corresponds to the 



adaptive processing portion 62 of the classifying 
adaptive processing portion 31, In Fig, 24, the 
register group 63 of the classifying adaptive 
processing portion 31 is omitted. 

A composite signal NTSC is supplied from an 
input terminal 141, A sub-carrier reproducing circuit 
142 extracts a sub-carrier from the supplied composite 
signal NTSC and reproduces it. In addition, the 
supplied composite signal NTSC is supplied to gain 
compensating circuits 143, 144, and 145. The gain 
compensating circuit 143 performs a gain compensation 
for converting the level of the composite signal NTSC 
into the level of the luminance signal Y. The signal 
that is output from the gain compensating circuit 143 
is the NTSC-Y signal and supplied to the classifying 
circuit 150. 

Likewise, the gain compensating circuit 144 
compensates the gain. A multiplying circuit 148 
multiplies a sub-carrier that has been phase- shifted by 
-123 degrees against the sub-carrier reproduced by the 
sub -carrier reproducing circuit 142 by the output of 
the gain compensating circuit 144 and thereby generates 
the NTSC-U signal. The NTSC-U signal is supplied to a 
classifying circuit 151. 

Likewise, the gain compensating circuit 145 
compensates the gain. A multiplying circuit 149 
multiplies a sub-carrier that has been phase- shifted by 
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-33 degrees against the sub-carrier reproduced by the 
sub -carrier reproducing circuit 142 by the output of 
the gain compensating circuit 145 and thereby generates 
the NTSC-V signal. The NTSC-V signal is supplied to a 
classifying circuit 152. 

The classifying circuit 150 outputs class 
code P and data B generated with the supplied NTSC-Y 
signal to a predictive filter 153. The predictive 
filter 153 reads a filter coefficient corresponding to 
the supplied class code P. The predictive filter 153 
calculates the filter coefficient and the supplied data 
B and thereby generates the luminance signal Y. The 
generated luminance signal Y is obtained from an output 
terminal 156. 

The classifying circuit 151 outputs class 
code P and data B generated with the supplied NTSC-U 
signal to a predictive filter 154. The predictive 
filter 154 reads a filter coefficient corresponding to 
the supplied class code P, calculates the filter 
coefficient and the supplied data B, and thereby 
generates the color difference signal U. The generated 
color difference signal U is obtained from an output 
terminal 157. 

Likewise, the classifying circuit 152 outputs 
class code P and data B generated with the supplied 
NTSC-V signal to a predictive filter 155. The color 
difference signal is obtained from the predictive 



filter 155 through an output terminal 158. In such a 
manner, the component signal (Y, U, V) can be obtained. 

Next, with reference to Figs. 25 and 26, 
class taps and predictive taps used in the process mode 
(3) will be described. As shown in Figs. 25A and 25B, 
when a considered pixel is pixel VO, class taps are 
composed of pixel VO, pixels VA and VB on the upper and 
lower lines of pixel VO at the corresponding positions 
of pixel VO in the field of pixel VO (0 field), pixels 
VC and VD that are apart from pixel VO by one pixel in 
the horizontal direction, and pixel VE of the preceding 
frame (-1 frame = -2 frame) corresponding to pixel VO. 
The reason why pixels VC and VD that are apart from 
pixel VO by one pixel in the horizontal direction are 
used is to synchronize the phase. 

On the other hand, as shown in Fig. 26, 
predictive taps are composed of pixel VO, pixels that 
immediately surround pixel VO in the field of pixel VO, 
the above-mentioned pixels VC and VD, and all the 
corresponding pixels of the preceding frame of the 
field of pixel VO. Since a coefficient is obtained in 
consideration of the difference of phases of pixels in 
the learning process (that will be described later), 
predictive taps can be extracted from pixels that have 
different phases. 

Next, the learning process in the process 
mode (3) will be described. Fig. 27 shows an example 



of the structure of the classifying adaptive processing 
portions 31-1, 31-2, and 31-3 that perform a learning 
process in the process mode (3). In the structure 
shown in Fig. 27, the thin -out circuit 91 of the 
5 structure shown in Fig. 11 is not used. Instead, an 

NTSC encoder 191 is disposed. In Fig. 27, similar 
portions to those in Fig. 11 are denoted by similar 
reference ntamerals and the detailed description thereof 
will be omitted. 
10 Predetermined luminance signal Y and color 

difference signals U and V are supplied to the NTSC 
^2 encoder 191. The NTSC encoder 191 encodes the input 

^ signals into a composite signal NTSC. The composite 

H signal NTSC is supplied to a classifying portion 92 and 

£3.5 a predictive tap extracting circuit 93. In other words, 

H in such a structure, with the composite signal NTSC, a 

ry considered pixel is classified and predictive taps are 

J5 extracted. In addition, the luminance signal Y and 

^" color difference signals U and V are supplied to a 

20 teacher data extracting circuit 95. In the structure, 

with teacher signal that are the luminance signal and 
color difference signals U and V, using the composite 
signal NTSC, a predictive coefficient set is generated. 

A predictive coefficient set is generated for 
25 each of the luminance signal Y and color difference 

signals U and V. The predictive coefficient set 
corresponding to the luminance signal Y is applied for 
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the classifying adaptive processing portion 31-1 
(namely, the classifying circuit 150 and the predictive 
filter 153). The predictive coefficient set 
corresponding to the color difference signal U is 
applied for the classifying adaptive processing portion 
31-2 (neimely, the classifying circuit 151 and the 
predictive filter 154). The predictive coefficient set 
corresponding to the color difference signal V is 
applied for the classifying adaptive processing portion 
31-3 (namely, the classifying circuit 152 and the 
predictive filter 155). 

Fig. 28 shows an example of the structure of 
the data processing apparatus 1 that converts the 
aspect ratio in the above -described process mode (4). 
In this example, the data processing portion 12 
increases the number of pixels only in the horizontal 
direction of a picture corresponding to picture data as 
input data (hereinafter referred to as input picture) 
and changes the aspect ratio. 

In this example, with three chronologically 
successive pixels (three pixels arranged successively 
in the horizontal direction) , four chronologically 
successive pixels (four pixels arranged successively in 
the horizontal direction) of a picture (hereinafter 
referred to as output picture) corresponding to picture 
data as output data are predicted. In this case, 
pixels of an output picture are newly predictively 
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created. In other words, in this example, an output 
picture is generated in the state that the ratio of the 
number of pixels in the horizontal direction of the 
input picture to the number of pixel in the horizontal 
5 direction of the output picture is 3 to 4. 

The function controlling portion 11 outputs a 
control signal corresponding to an input command to the 
data processing portion 12. The data processing 
portion 12 selects a bank corresponding to the process 
10 mode (4) corresponding to the control signal supplied 

from the data processing portion 12. The data 
processing portion 12 executes a classifying adaptive 
CO process for generating four pixels with three pixels 

N corresponding to a coefficient set of the selected bank. 

£315 The data processing portion 12 performs the classifying 

1^ adaptive process for each of the four pixels. Thus, 

fy the selector 140 of the post-processing portion 14 

selects the multiplexing portion 14 IC that performs a 
multiplexing process corresponding to a control signal 
20 supplied from the function controlling portion 11. 

The multiplexing portion 141C multiplexes 
picture data supplied from the data processing portion 
12. In other words, the multiplexing portion 141C 
outputs an output picture of which the aspect ratio of 
25 the input picture was changed as output data to an 

external device. 

Fig. 29 shows an example of the connections 
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of the data processing portion 12 in the process mode 
(4). In this example, in the input data processing 
portion 21, the four classifying adaptive processing 
portions 31-1 to 31-4 are used. An input picture is 
supplied to the classifying adaptive processing 
portions 31-1 to 31-4. 

The input data processing portion 21 performs 
a predetermined classifying adaptive process for the 
input picture and outputs picture data of which the 
ratio of the number of pixels in the horizontal 
direction of an input picture to the number of pixels 
in the horizontal direction of an output picture is 3 
to 4 to the selector 22. 

The selector 22 supplies picture data 
supplied form the input data processing portion 21 to 
the post -processing portion 14. As was described above, 
in the post-processing portion 14, the selector 140 
selects the multiplexing portion 14 IC corresponding to 
a control signal supplied form the function controlling 
portion 11. The picture data supplied to the post- 
processing portion 14 is supplied to the multiplexing 
portion 141C. The multiplexing portion 141C 
multiplexes the picture data supplied from the post- 
processing portion 14 and outputs the multiplexed 
picture data as output picture data. 

The intermediate data processing portion 23 
and the output data processing portion 24 are not used 



in the process mode (4), 

Next, with reference to Fig. 30, the 
predictive process of the classifying adaptive 
processing portion 31 of the input data processing 
5 portion 21 in the process mode (4) will be described. 

Figs. 30A and SOB schematically show pixels of an input 
picture and pixels of an output picture. 

The classifying adaptive processing portion 
31-1 performs a classifying adaptive process in such a 
10 manner that a pixel of an output picture that has a 

first phase relation to a pixel of an input picture 
f (namely, a pixel arranged on the same vertical line as 

a pixel of an input picture) is designated as a 
considered pixel and predicts the pixel value of the 
!15 considered pixel. In the example shown in Fig. 30, 

pixels Pbl and Pb5 that compose an output picture and 
pixels Pal and Pa4 that compose an input picture and 
5 that correspond to pixels pbl and pb5 are used as 

considered pixels . 
20 The classifying adaptive processing portion 

31-2 performs a classifying adaptive process in such a 
manner that a pixel that is immediately chronologically 
preceded by a pixel whose pixel value has been 
predicted by the classifying adaptive processing 
25 portion 31-1 (namely, a pixel having a second phase 

relation to a pixel of an input picture) is designated 
as a considered pixel and predicts the pixel value of 
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the considered pixel. In the example shown in Fig. 30B, 
pixel Pbl is a pixel whose pixel value is predicted by 
the classifying adaptive processing portion 31-1. 
Pixel Pb2 arranged adjacent to pixel Pbl is a 
considered pixel designated by the classifying adaptive 
processing portion 31-2, 

The classifying adaptive processing portion 
31-3 performs a classifying adaptive process in such a 
manner that a pixel that is immediately chronologically 
preceded by a pixel whose pixel value is predicted by 
the classifying adaptive processing portion 31-2 
(namely, a pixel having a third phase relation to a 
pixel of an input picture) is designated as a 
considered pixel and predicts the pixel value of the 
considered pixel. In the example shown in Fig. 30B, 
pixel Pb2 is a pixel whose pixel value is predicted by 
the classifying adaptive processing portion 31-2. 
Pixel Pb3 arranged adjacent to pixel Pb2 is a 
considered pixel designated by the classifying adaptive 
processing portion 31-3. 

The classifying adaptive processing portion 
31-4 performs a classifying adaptive process in such a 
manner that a pixel (having a fourth phase relation to 
a pixel of an input picture) that is immediately 
chronologically preceded by a pixel (in the example 
shown in Fig. 30B, pixel Pb3) whose pixel value is 
predicted by the classifying adaptive processing 



portion 31-3 is designated as a considered pixel and 
predicts the pixel value of the considered pixel. In 
the example shown in Fig. 30B, pixel Pb3 is a pixel 
whose pixel value is predicted by the classifying 
adaptive processing portion 31-3. Pixel Pb4 arranged 
adjacent to pixel Pb3 is a considered pixel designated 
by the classifying adaptive processing portion 31-4. 

In other words, since the number of pixels in 
the horizontal direction is increased in the ratio of 3 
to 4, there are four types of phase relations in the 
horizontal relation of a pixel (considered pixel) of an 
output picture and a pixel of an input picture. Each 
of the classifying adaptive processing portions 31-1 to 
31-4 performs a classifying adaptive process 
corresponding to one of the four types of phase 
relations and supplies the generated pixel to the 
selector 22. 

The selector 22 supplies picture data (pixel) 
supplied from the classifying adaptive processing 
portion 31 to the post-processing portion 14. As was 
described above, in the post -processing portion 14, the 
selector 14 supplies the supplied picture data to the 
multiplexing portion 141C corresponding to a control 
signal supplied from the function controlling portion 
11. The multiplexing portion 141C multiplexes the 
picture data and outputs the multiplexed picture data 
to an external device. In such a manner, the aspect 
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ratio is changed. 

In the process mode (4), by increasing the 
nximber of pixels in the horizontal scanning direction, 
the aspect ration is changed. Likewise, by decreasing 
the number of pixels in the horizontal scanning 
direction, the aspect ratio can be changed. 

Fig. 31 shows an example of the structure of 
the data processing apparatus 1 that generates a 
picture having a different resolution in the above- 
described process mode (5). In the example, the data 
processing portion 12 generates picture data 
corresponding to a plurality of pictures having 
different resolutions with SD picture data as input 
data supplied from an external device and outputs the 
generated picture data as output data to an external 
device . 

Fig. 32 shows an excimple of the connections 
of the data processing portion 12 in the process mode 
(5). In the example, in the input data processing 
portion 21, the classifying adaptive processing 
portions 31-1 and 31-2 are used. SD picture data as 
input data is supplied to the classifying adaptive 
processing portions 31-1 and 31-2. 

The classifying adaptive processing portion 
31 performs a classifying adaptive process for supplied 
SD picture data and supplies the generated picture data 
having a first resolution to the selector 22. 
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The classifying adaptive processing portion 
31-2 performs a classifying adaptive process for the 
supplied SD picture data and supplies the generated 
picture data having a second resolution to the selector 
22. 

The selector 22 supplies picture data having 
the first resolution supplied from the classifying 
adaptive processing portion 31-1 of the input data 
processing portion 21 to the classifying adaptive 
processing portions 51-1 and 51-2 of the output data 
processing portion 24. 

The selector 22 supplies the picture data 
having the second resolution supplied from the 
classifying adaptive processing portion 31-2 of the 
input data processing portion 21 to the classifying 
adaptive processing portions 51-3 and 51-4 of the 
output data processing portion 24. 

In this example, in the output data 
processing portion 24, the classifying adaptive 
processing portions 51-1 to 51-4 are used. 

The classifying adaptive processing portion 
51-1 performs a classifying adaptive process for the 
picture data having the first resolution supplied 
through the selector 22 and generated by the 
classifying adaptive processing portion 31-1 and 
outputs the generated picture data having the third 
resolution as output data. 



The classifying adaptive processing portion 
51-2 performs a classifying adaptive process for the 
picture data having the first resolution supplied 
through the selector 22 and generated by the 
classifying adaptive processing portion 31-1 and 
outputs the generated picture data having the fourth 
resolution as output data. 

The classifying adaptive processing portion 
51-3 performs a classifying adaptive process for the 
picture data having the second resolution supplied 
through the selector 22 and generated by the 
classifying adaptive processing portion 31-2 and 
outputs the generated picture data having the fifth 
generation as output data. 

The classifying adaptive processing portion 
51-4 performs a classifying adaptive process for the 
picture data having the second resolution supplied 
through the selector 22 and generated by the 
classifying adaptive processing portion 31-2 and 
outputs the generated picture data having the sixth 
resolution as output data. 

In the process mode (5), a plurality of 
pictures having different resolutions can be generated. 
Thus, for example, multi-windows having different sizes 
can be displayed. 

The classifying adaptive process of each 
classifying adaptive processing portion of the input 
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data processing portion 21 and the output data 
processing portion 24 may be the same as that of the 
above -described data processing apparatus 1 in the 
process mode (1). In addition, in the process mode (5), 
5 the same class taps and predictive taps as those in the 

above -described process mode (1) can be used. 

Fig. 33 shows an example of the structure of 
the data processing apparatus 1 that converts the frame 
rate in the above -described process mode (6). In this 
10 case, the data processing apparatus 1 converts the 

frame rate of a picture that is input as input data. 

The function controlling portion 11 supplies 
W a control signal corresponding to a command that is 

H input in the process mode (6) to the data processing 

C3.5 portion 12. The data processing portion 12 selects a 

bank for the process mode (6) corresponding to the 
ry control signal supplied from the function controlling 

J5 portion 11. The data processing portion 12 executes a 

classifying adaptive process for generating a new frame 
20 with picture data as input data corresponding to a 

coefficient set of the selected bank. In the process 
mode (6), the post-processing portion 14 selects the 
multiplexing portion 14 IC corresponding to the control 
signal supplied from the function controlling portion 
25 11 so as to multiplex a frame that is newly generated 

by the data processing portion 12. 

A multiplexing portion 106 chronologically 
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multiplexes picture data (as frames) supplied from the 
data processing portion 12 and outputs the multiplexed 
data as output data to the outside. 

Fig. 34 shows an example of the connections 
of the data processing portion 12 in the process mode 
(6). In the example, in the input data processing 
portion 21, the classifying adaptive processing 
portions 31-1 and 31-2 are used. A picture as input 
data is input to each of the classifying adaptive 
processing portions 31-1 and 31-2. 

As shown in Fig. 35A, it is assumed that the 
input picture has a frame structure with a period of Ta. 
On the other hand, as shown in Fig. 35B, it is assumed 
that an output picture whose freune rate has been 
converted has a frame structure with a period of Ta/2. 
In other words, in this example, the frame rate is 
doubled. 

In the classifying adaptive processing 
portion 31-1, the multiplex portion 141C performs a 
classifying adaptive process for generating a frame of 
an output picture that is chronologically preceded by a 
frame that composes an input picture. In reality, the 
classifying adaptive processing portions 31-1 and 31-2 
treats a frame in the unit of a pixel. In the 
classifying adaptive process of the classifying 
adaptive processing portion 31-1, a frame that composes 
an output picture is generated. In the example shown 



in Fig. 35, as shown in Fig. 35B, frames B-2, B-4, and 
B-6 (white frames in Fig. 35B) of an output picture 
chronologically followed by frames A-1 to A- 3 of an 
input picture shown in Fig. 35A are generated by the 
5 classifying adaptive processing portion 31-1 (see Fig. 

35B) . The classifying adaptive processing portion 31-1 
supplies the generated frcimes to the selector 22. 

The classifying adaptive processing portion 
31-2 performs a classifying adaptive process for 
10 generating a frame that composes an output picture and 

that is chronologically followed by a frame that 
Hp composes an input picture by the multiplexing portion 

M 14 IC disposed downstream thereof. In the classifying 

M adaptive process of the classifying adaptive processing 

t%5 portion 31-2, another frame that composes the output 

picture is generated. In the example shown in Fig. 35, 
as shown in Fig. 35B, frames B-1, B-3, and B-5 (hatched 
^5? frames shown in Fig. 35B) of an output picture 

chronologically followed by frames A-1 to A- 3 of an 
20 input picture shown in Fig. 35A are generated (see Fig. 

35B) . The classifying adaptive processing portion 31-2 
supplies the generated frames to the selector 22. 

The selector 22 supplies the frames supplied 
from the classifying adaptive processing portions 31-1 
25 and 31-2 to the post-processing portion 14. The post- 

processing portion 14 supplies the frames selected by 
the selector 140 corresponding to the control signal 
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supplied from the function controlling portion 11 to 
the multiplexing portion 141C. The multiplexing 
portion 141C multiplexes the supplied frcunes in the 
rule as was described with reference to Fig* 35B and 
5 outputs the multiplexed data as output data to an 

external device. In the process mode (6), in such a 
manner, the frame rate is converted. 

Fig. 36 shows an example of class tap 
extraction information in the process mode (6). Next, 
10 the class tap extraction information will be described 

Q with reference to Fig. 35A. In frame A- 2 that contains 

a considered pixel, 3 pixels x 3 pixels whose center 
S pixel is the considered pixel are extracted as class 

taps. In frames A-1 and A- 3 that are immediately 
^Is preceded and followed by frame A- 2, pixels 

f" corresponding to those extracted as the class taps in 

n frame A- 2 are selected as class taps. In other words, 

O in the process mode (6), class taps are composed of 3 

pixels X 3 pixels x 3 frames . 
20 Predictive taps (not shown) include for 

example frame A- 2 that has a considered pixel. A total 
of five frames composed of frame A-2, two frames 
preceded by frame A-2, and two frames followed by frame 
A-2 are extracted. In each of the five frames, 5 
25 pixels X 5 pixels wider than class taps by one pixel 

are extracted. In other words, in the process mode (6), 
predictive taps are composed of for example 5 pixels x 



5 pixels X 5 frames . 

The classifying adaptive process of the 
classifying adaptive processing portions 31-1 and 31-2 
of the input data processing portion 21 may be the same 
as that of the data processing apparatus 1 in the 
process mode (1). 

In the above description, an example of which 
the present invention is applied to picture data was 
explained. However, the present invention is not 
limited to such an example. In other words, the 
present invention can be applied to other data such as 
audio data. 

Moreover, in the above description, the data 
processing apparatus 1 accomplishes the above -de scribed 
process modes (1) to (6). However, the present 
invention is not limited to such an example. In other 
words, in the data processing apparatus 1, with a 
predetermined predictive coefficient set, a 
predetermined class tap structure, and a predetermined 
predictive tap structure, processes other than the 
process modes (1) to (6) can be accomplished. 



